A Proposal for a Data-Driven Approach to the Influence of Music on Heart Dynamics.

Electrocardiographic signals (ECG) and heart rate viability measurements (HRV) provide information in a range of specialist fields, extending to musical perception. The ECG signal records heart electrical activity, while HRV reflects the state or condition of the autonomic nervous system. HRV has been studied as a marker of diverse psychological and physical diseases including coronary heart disease, myocardial infarction, and stroke. HRV has also been used to observe the effects of medicines, the impact of exercise and the analysis of emotional responses and evaluation of effects of various quantifiable elements of sound and music on the human body. Variations in blood pressure, levels of stress or anxiety, subjective sensations and even changes in emotions constitute multiple aspects that may well-react or respond to musical stimuli. Although both ECG and HRV continue to feature extensively in research in health and perception, methodologies vary substantially. This makes it difficult to compare studies, with researchers making recommendations to improve experiment planning and the analysis and reporting of data. The present work provides a methodological framework to examine the effect of sound on ECG and HRV with the aim of associating musical structures and noise to the signals by means of artificial intelligence (AI); it first presents a way to select experimental study subjects in light of the research aims and then offers possibilities for selecting and producing suitable sound stimuli; once sounds have been selected, a guide is proposed for optimal experimental design. Finally, a framework is introduced for analysis of data and signals, based on both conventional as well as data-driven AI tools. AI is able to study big data at a single stroke, can be applied to different types of data, and is capable of generalisation and so is considered the main tool in the analysis.

Judgement of valence of musical sounds by hand and by heart, a machine learning paradigm for reading the heart.

The intention of the experiment is to investigate whether different sounds have influence on heart signal features in the situation the observer is judging the different sounds as positive or negative. As the heart is under (para)sympathetic control of the nervous system this experiment could give information about the processing of sound stimuli beyond the conscious processing of the subject. As the nature of the influence on the heart signal is not known these signals are to be analysed with AI/machine learning techniques. Heart rate variability (HRV) is a variable derived from the R-R interval peaks of electrocardiogram which exposes the interplay between the sympathetic and parasympathetic nervous system. In addition to its uses as a diagnostic tool and an active part in the clinic and research domain, the HRV has been used to study the effects of sound and music on the heart response; among others, it was observed that heart rate is higher in response to exciting music compared with tranquilizing music while heart rate variability and its low-frequency and high-frequency power are reduced. Nevertheless, it is still unclear which musical element is related to the observed changes. Thus, this study assesses the effects of harmonic intervals and noise stimuli on the heart response by using machine learning. The results show that noises and harmonic intervals change heart activity in a distinct way; e.g., the ratio between the axis of the ellipse fitted in the Poincaré plot increased between harmonic intervals and noise exposition. Moreover, the frequency content of the stimuli produces different heart responses, both with noise and harmonic intervals. In the case of harmonic intervals, it is also interesting to note how the effect of consonance quality could be found in the heart response.

Can the application of certain music information retrieval methods contribute to the machine learning classification of electrocardiographic signals?

The electrocardiogram is traditionally used to diagnose a large number of heart pathologies. Research to improve the readability and classification of cardiac signals includes studies geared toward sonification of the electrocardiographic signal and others involving features related to music processing, such as Mel-frequency cepstral coefficients. In terms of music processing features, this study seeks to use music information retrieval (MIR) features as electrocardiographic signal descriptors. The study compares the discriminatory capability of the introduced features in relation to standard groups such as heart rate variability, wavelet transform, descriptive statistics, Mel coefficients and fractal analysis, evaluated using classification algorithms; the signals analyzed were extracted from public databases. The group of features extracted from wavelet transform and the MIR group showed a high level of discrimination; the best representation of the ECG signals in the study was achieved in most cases by the MIR features. Moreover, a correlation coefficient higher than 0.8 was found between a number of MIR and other feature groups, indicating a likely relationship between the electrocardiographic signals and MIR features. These results suggest the feasibility of representing the analyzed signals by music information retrieval descriptors, giving the potential to consider these electrocardiographic signals as analogues to musical signals.

Different Types of Sounds and Their Relationship With the Electrocardiographic Signals and the Cardiovascular System - Review.

Background: For some time now, the effects of sound, noise, and music on the human body have been studied. However, despite research done through time, it is still not completely clear what influence, interaction, and effects sounds have on human body. That is why it is necessary to conduct new research on this topic. Thus, in this paper, a systematic review is undertaken in order to integrate research related to several types of sound, both pleasant and unpleasant, specifically noise and music. In addition, it includes as much research as possible to give stakeholders a more general vision about relevant elements regarding methodologies, study subjects, stimulus, analysis, and experimental designs in general. This study has been conducted in order to make a genuine contribution to this area and to perhaps to raise the quality of future research about sound and its effects over ECG signals. Methods: This review was carried out by independent researchers, through three search equations, in four different databases, including: engineering, medicine, and psychology. Inclusion and exclusion criteria were applied and studies published between 1999 and 2017 were considered. The selected documents were read and analyzed independently by each group of researchers and subsequently conclusions were established between all of them. Results: Despite the differences between the outcomes of selected studies, some common factors were found among them. Thus, in noise studies where both BP and HR increased or tended to increase, it was noted that HRV (HF and LF/HF) changes with both sound and noise stimuli, whereas GSR changes with sound and musical stimuli. Furthermore, LF also showed changes with exposure to noise. Conclusion: In many cases, samples displayed a limitation in experimental design, and in diverse studies, there was a lack of a control group. There was a lot of variability in the presented stimuli providing a wide overview of the effects they could produce in humans. In the listening sessions, there were numerous examples of good practice in experimental design, such as the use of headphones and comfortable positions for study subjects, while the listening sessions lasted 20 min in most of the studies.

Eye movements in scene perception while listening to slow and fast music.

To date, there is insufficient knowledge of how visual exploration of outdoor scenes may be influenced by the simultaneous processing of music. Eye movements during viewing various outdoor scenes while listening to music at either a slow or fast tempo or in silence were measured. Significantly shorter fixations were found for viewing urban scenes com-pared with natural scenes, but there was no interaction between the type of scene and the acoustic conditions. The results revealed shorter fixation durations in the silent control condition in the range 30 ms, compared to both music conditions but, in contrast to previ-ous studies, these differences were non-significant. Moreover, we did not find differences in eye movements between music conditions with a slow or fast tempo. It is supposed that the type of musical stimuli, the specific tempo, the specific experimental procedure, and the engagement of participants in listening to background music while processing visual information may be important factors that influence attentional processes, which are mani-fested in eye-movement behavior.

Tempo and walking speed with music in the urban context.

The study explored the effect of music on the temporal aspects of walking behavior in a real outdoor urban setting. First, spontaneous synchronization between the beat of the music and step tempo was explored. The effect of motivational and non-motivational music (Karageorghis et al., 1999) on the walking speed was also studied. Finally, we investigated whether music can mask the effects of visual aspects of the walking route environment, which involve fluctuation of walking speed as a response to particular environmental settings. In two experiments, we asked participants to walk around an urban route that was 1.8 km in length through various environments in the downtown area of Hradec Králové. In Experiment 1, the participants listened to a musical track consisting of world pop music with a clear beat. In Experiment 2, participants were walking either with motivational music, which had a fast tempo and a strong rhythm, or with non-motivational music, which was slower, nice music, but with no strong implication to movement. Musical beat, as well as the sonic character of the music listened to while walking, influenced walking speed but did not lead to precise synchronization. It was found that many subjects did not spontaneously synchronize with the beat of the music at all, and some subjects synchronized only part of the time. The fast, energetic music increases the speed of the walking tempo, while slower, relaxing music makes the walking tempo slower. Further, it was found that listening to music with headphones while walking can mask the influence of the surrounding environment to some extent. Both motivational music and non-motivational music had a larger effect than the world pop music from Experiment 1. Individual differences in responses to the music listened to while walking that were linked to extraversion and neuroticism were also observed. The findings described here could be useful in rhythmic stimulation for enhancing or recovering the features of movement performance.

Encouraging spontaneous synchronisation with D-Jogger, an adaptive music player that aligns movement and music.

In this study we explore how music can entrain human walkers to synchronise to the musical beat without being instructed to do so. For this, we use an interactive music player, called D-Jogger, that senses the user's walking tempo and phase. D-Jogger aligns the music by manipulating the timing difference between beats and footfalls. Experiments are reported that led to the development and optimisation of four alignment strategies. The first strategy matched the music's tempo continuously to the runner's pace. The second strategy matched the music's tempo at the beginning of a song to the runner's pace, keeping the tempo constant for the remainder of the song. The third alignment starts a song in perfect phase synchrony and continues to adjust the tempo to match the runner's pace. The fourth and last strategy additionally adjusts the phase of the music so each beat matches a footfall. The first two strategies resulted in a minor increase of steps in phase synchrony with the main beat when compared to a random playlist, the last two strategies resulted in a strong increase in synchronised steps. These results may be explained in terms of phase-error correction mechanisms and motor prediction schemes. Finding the phase-lock is difficult due to fluctuations in the interaction, whereas strategies that automatically align the phase between movement and music solve the problem of finding the phase-locking. Moreover, the data show that once the phase-lock is found, alignment can be easily maintained, suggesting that less entrainment effort is needed to keep the phase-lock, than to find the phase-lock. The different alignment strategies of D-Jogger can be applied in different domains such as sports, physical rehabilitation and assistive technologies for movement performance.

Activating and relaxing music entrains the speed of beat synchronized walking.

Inspired by a theory of embodied music cognition, we investigate whether music can entrain the speed of beat synchronized walking. If human walking is in synchrony with the beat and all musical stimuli have the same duration and the same tempo, then differences in walking speed can only be the result of music-induced differences in stride length, thus reflecting the vigor or physical strength of the movement. Participants walked in an open field in synchrony with the beat of 52 different musical stimuli all having a tempo of 130 beats per minute and a meter of 4 beats. The walking speed was measured as the walked distance during a time interval of 30 seconds. The results reveal that some music is 'activating' in the sense that it increases the speed, and some music is 'relaxing' in the sense that it decreases the speed, compared to the spontaneous walked speed in response to metronome stimuli. Participants are consistent in their observation of qualitative differences between the relaxing and activating musical stimuli. Using regression analysis, it was possible to set up a predictive model using only four sonic features that explain 60% of the variance. The sonic features capture variation in loudness and pitch patterns at periods of three, four and six beats, suggesting that expressive patterns in music are responsible for the effect. The mechanism may be attributed to an attentional shift, a subliminal audio-motor entrainment mechanism, or an arousal effect, but further study is needed to figure this out. Overall, the study supports the hypothesis that recurrent patterns of fluctuation affecting the binary meter strength of the music may entrain the vigor of the movement. The study opens up new perspectives for understanding the relationship between entrainment and expressiveness, with the possibility to develop applications that can be used in domains such as sports and physical rehabilitation.

Four applications of embodied cognition.

This article presents the views of four sets of authors, each taking concepts of embodied cognition into problem spaces where the new paradigm can be applied. The first considers consequences of embodied cognition on the legal system. The second explores how embodied cognition can change how we interpret and interact with art and literature. The third examines how we move through architectural spaces from an embodied cognition perspective. And the fourth addresses how music cognition is influenced by the approach. Each contribution is brief. They are meant to suggest the potential reach of embodied cognition, increase the visibility of applications, and inspire potential avenues for research.

Walking on music.

The present study focuses on the intricate relationship between human body movement and music, in particular on how music may influence the way humans walk. In an experiment, participants were asked to synchronize their walking tempo with the tempo of musical and metronome stimuli. The walking tempo and walking speed were measured. The tempi of the stimuli varied between 50 and 190 beats per minute. The data revealed that people walk faster on music than on metronome stimuli and that walking on music can be modeled as a resonance phenomenon that is related to the perceptual resonance phenomenon as described by Van Noorden and Moelants (Van Noorden, L., & Moelants, D. (1999). Resonance in the perception of musical pulse. Journal of New Music Research, 28, 43-66).